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1.0  SUMMARY 


Communication  remains  a  critical  component  to  mission  success.  With  the  shift  towards  network-centric 
warfare,  standard  radio  communication  needs  to  meet  the  needs  of  today’s  warfighter.  A  net-centric 
communication  management  suite  called  Multi-Modal  Communication  (MMC)  had  been  developed.  This 
integrated  system  captures,  displays,  records,  and  archives  radio  and  text-based  communication  to  better 
equip  the  warfighters.  This  study  examined  the  performance  associated  with  monitoring  communication 
channels,  with  access  to  different  tools.  Communication  performance  was  analyzed  in  regard  to  message 
detection,  response  accuracy,  and  time.  Data  showed  that  MMC  provides  a  balance  between  the  speed  of 
radio  listening  and  the  accuracy  and  data-capturing  capabilities  of  chat  displays.  MMC  can  be  a  beneficial 
tool  to  operators  in  its  ability  to  increase  intelligibly,  while  providing  a  persistent,  searchable  display  that 
reduces  perceived  mental  workload. 


2.0  INTRODUCTION 

Chief  of  Naval  Operations  Admiral  Jay  Johnson  stated,  “the  fundamental  shift  from  what  we  call  platform¬ 
centric  warfare  to  something  we  call  network-centric  warfare  will  prove  to  be  the  most  important  revolution  in 
military  affairs  in  the  past  200  years”  (Cited  from  Cebrowski  &  Garstka,  1998).  The  motivation  for  this 
transformation  is  to  increase  mission  effectiveness  with  the  use  of  vast  networks  of  state-of-the-art  sensor  and 
weapon  platforms.  The  theory  is  that  instant  access  to  information  from  dispersed  resources  will  increase  the 
speed  of  command  and  self-synchronization  of  all  active  operators,  thus  improving  military  capabilities 
(Cebrowski  &  Garstka,  1998).  These  benefits  are  a  result  of  advanced  collaborative  tools  that  allow  distributed 
teams  to  communicate  and  share  information,  thus  increasing  each  other’s  situation  awareness  of  the  ever- 
changing  mission  (Vidulich,  Bolia,  &  Nelson,  2003).  It  is  important  to  point  out  that  although  advanced 
collaborative  tools  will  aid  in  the  sharing  of  information,  accurate  communication  between  operators  can  never 
be  replaced  (Nelson,  Bolia,  Vidulich,  &  Langhome,  2004).  Communication  will  continue  to  be  the  central  tool 
for  Command  and  Control  (C2)  operators.  However,  current  radios  are  still  analog  and  are  not  fully  integrated 
into  modem  day  collaborative  networks. 

The  effects  of  dated  technology  can  be  seen  in  situations  confronting  C2  operators  who  monitor 
communications  from  multiple  assets  to  build  a  real-time  picture  of  the  dynamic  battlefield  and  direct  them  as 
needed  to  carry  out  the  mission.  This  communication-intensive  environment  imposes  high  workload  on  the 
operators,  who  typically  monitor  and  transmit  on  eight  or  more  simultaneous  channels  (Bolia,  2003).  A  major 
cause  of  the  high  mental  workload  is  the  issue  that  multiple  operators  can  speak  at  the  same  time,  thus  reducing 
the  intelligibility  of  essential  messages.  An  additional  problem  is  the  transient  nature  of  radio  transmissions,  in 
that  the  recipient  has  one  chance  to  extract  crucial  infomiation  or  is  required  to  request  that  it  is  repeated.  Not 
only  is  missed  information  a  problem  during  the  mission,  but  the  operators  must  take  detailed  notes  to  ensure 
the  correct  information  is  relayed  to  the  appropriate  asset.  This  added  step  to  relay  the  information  to  the  correct 
asset  opens  up  the  possibility  for  error  and  may  place  the  mission  in  jeopardy. 

In  an  attempt  to  better  equip  C2  operators  with  collaborative  tools  that  could  aid  in  their  mission,  a  survey  of  air 
battle  managers  from  the  United  States  Air  Force,  United  States  Navy,  and  the  Royal  Australian  Air  Force 
assessed  the  perceived  usefulness  of  current  commercial-off-the-shelf  collaborative  tools  (Nelson,  Bolia, 
Vidulich,  &  Langhome,  2004).  The  two  highest  ratings  for  most  potential  technologies  were  data  capture/replay 
tools  and  chat/messaging  tools.  Based  on  this  information  and  research  on  communication  intelligibility, 
researchers  at  the  Battlespace  Acoustics  Branch  in  the  Air  Force  Research  Laboratory  developed  a  network¬ 
centric  communication  management  suite  to  help  improve  communication  performance  for  network-centric 
warfare.  This  program,  called  Multi-Modal  Communication  (MMC),  combines  mature  communication 

l 

Distribution  A:  Approved  for  public  release;  distribution  unlimited. 

88  ABW  Cleared  04/29/2011;  88ABW-201 1-2499. 


technologies  into  an  integrated  communication  system,  see  Figure  1.  This  combination  fosters  collaborative 
decision-making  by  providing  operators  with  full  access  to  voice  and  text-chat  communication  data  from  the 
distributed  operator. 

MMC  allows  operators  to  manage  communication  from  voice  as  well  as  text-based  systems  in  a  single  dynamic 
display.  To  combat  the  issue  with  radio  intelligibility  due  to  multiple  operators  talking,  MMC  spatially  separates 
each  of  the  voice  channels  to  virtual  locations  around  the  operator  via  their  headphones.  Digital  fdters,  called 
head -related  transfer  functions,  take  advantage  of  binaural  cues  that  normally  occur  when  competing  talkers  are 
spatially  separated,  and  display  the  sounds  over  headphones  in  a  similar  manner.  These  types  of  auditory 
displays  can  produce  virtual  sounds  that  are  as  localizable  as  sounds  produced  in  the  free  field  and  have  been 
shown  to  increase  speaker  intelligibility  by  30-40%  for  headphone -based  multi-talker  communication,  and  to 
reduce  listeners’  perceived  mental  workload  (Bolia,  2003;  Bolia  &  Nelson,  2003;  Brungart  &  Simpson,  2005; 
Ericson,  Brungart,  &  Simpson,  2004;  Nelson,  Bolia,  Ericson,  &  McKinley  1998). 


Figure  1.  Two  MMC  displays.  1)  Spatial  location  of  audio  channel;  2)  Chat  input  field;  3)  Control  buttons;  4)  Playback, 
edit/annotate  controls,  and  flag  icon  tags;  5)  Flagged/Transcribed  text;  6)  Find  control;  7)  Keyword  highlight. 


As  stated  earlier,  a  major  problem  with  voice  communication  is  that  it  is  perishable  in  that  operators  only 
have  one  chance  to  extract  critical  information  from  a  transmission  and  retain  it  before  it  is  gone.  Two 
features  within  MMC  address  that  issue.  The  repeat  function  allows  operators  to  replay  the  last  10  seconds  of 
communication  to  clarify  any  missed  information.  The  other  feature  that  combats  this  issue  is  the  use  of 
automatic  speech  recognition  (ASR),  which  captures  and  displays  voice  communication  as  text,  thus  making 
a  once  perishable  message  persistent.  All  voice  communication  is  textually  displayed  and  time  stamped  to  aid 
operators  in  message  detection  and  recall.  In  addition  to  capturing  the  message  as  text,  the  audio  file  is  also 
captured  and  attached,  allowing  the  operator  to  select  the  text  and  relisten  to  the  original  message  or  group  of 
messages. 

MMC  also  accepts  and  transmits  chat  messages  from  most  multi-user  chat  programs.  MMC  follows  a  client- 
server  framework  where  a  user  sends  a  chat  message  via  the  client  to  a  server.  The  server  then  distributes  the 
chat  message  to  the  connected  clients,  which  stores  the  chat  data,  audio  messages  and  their  transcriptions  in 
time  order,  thus  maintaining  a  complete  log  of  all  communication  during  a  mission.  Operators  who  join  the 
communication  channels  after  the  start  of  the  mission,  who  have  been  disconnected  or  who  take  over  a  shift, 
now  will  have  access  to  all  verbal  and  written  messages,  thus  allowing  them  to  gain  situation  awareness  by 
reviewing  the  log.  Currently,  there  is  a  lengthy  exchange  between  the  oncoming  and  outgoing  operators  during 
shift  change.  Since  the  data  is  stored  as  text,  MMC  allows  operators  to  highlight  or  flag  important  transmissions 
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so  they  stand  out  for  easy  review.  In  addition,  the  flags  can  be  categorized  to  allow  messages  to  be  easily 
organized. 


The  information  being  passed  from  one  operator  to  another  can  be  a  matter  of  life  or  death,  thus  the  accuracy  of 
that  data  is  of  the  utmost  importance.  Given  that  ASR  is  not  always  perfect  or  that  there  can  be  typos  in  chat 
entries,  MMC  allows  messages  to  be  edited  to  reflect  accurate  information.  Human-  based  computation  can  be 
employed  here,  as  proofreaders  for  all  transcribed  messages  in  that  operators  not  directly  related  to  the  mission 
can  hear  and  read  transcribed  messages  and  make  necessary  corrections.  These  edits  then  can  be  reprocessed  to 
improve  the  ASR  program.  Another  advantage  of  displaying  transcribed  messages  is  that  they  can  be  formatted 
and  presented  to  the  operators  so  they  can  clearly  review  the  material.  An  additional  feature  in  MMC  to  help 
reduce  transmission  errors  is  the  ability  to  copy  a  message  and  paste  it  to  the  appropriate  channel  for 
dissemination. 

Operators  can  configure  MMC  to  alert  them  to  the  occurrence  of  keywords.  The  keywords  are  highlighted  to 
an  operator’s  preference  so  that  messages  containing  these  keywords  stand  out  and  are  clearly  visible.  In 
addition  to  highlighting  keywords,  MMC  provides  operators  with  the  full  capability  to  search  and  find  critical 
information.  The  “Find”  function  allows  them  to  search  for  keywords  or  flagged  items,  while  the  “History” 
function  provides  operators  with  the  ability  to  jump  to  specific  times  and  review  past  information. 

These  features  have  been  integrated  into  a  single  unified  system  to  allow  for  interoperability  of  verbal  and  chat 
communication.  The  idea  behind  the  design  of  MMC  is  to  give  operators  full  access  to  all  current  and  past 
information,  thus  allowing  them  to  make  quick  and  accurate  decisions  while  decreasing  the  mental  workload 
associated  with  a  C2  task. 

The  goal  for  this  study  was  to  assess  the  potential  utility  of  the  MMC  suite  as  a  network-centric 
communication-  management  tool  when  compared  to  radio  communication  with  or  without  3D  spatial  audio, 
and  chat.  It  was  expected  that  the  combination  of  communication  tools  would  aid  operators  in  their  ability  to 
quickly  and  accurately  detect  and  reply  to  critical  messages  as  well  as  reduce  their  perceived  mental  workload. 
In  addition  to  manipulating  Comm.  Format,  the  amount  of  information  each  participant  monitored  was  varied. 
The  Easy  message  detection  condition  did  not  require  working  memory  of  critical  (hostile)  information,  signal 
since  it  was  composed  of  the  call  sign  and  the  word  “Hostile”.  The  Hard  message  detection  condition  required 
operators  to  monitor  and  respond  with  all  of  the  hostile’s  location  information,  thus  requiring  more  of  a 
working  memory  load.  The  second  expectation  was  that  the  combination  of  communication  tools  would  aid 
operators  in  their  response  to  the  hard  message  detection  conditions.  Performance  was  measured  by  the  ability 
of  operators  to  detect  the  critical  messages,  the  accuracy  of  their  responses,  and  the  time  they  took  to  reply. 
Finally,  it  was  expected  that  the  use  of  communication  tools  would  reduce  the  perceived  mental  workload, 
associated  with  the  MMC  task  as  compared  to  the  other  more  traditional  communication  formats  as  described 
below. 
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3.0  METHODS,  ASSUMPTIONS,  AND  PROCEDURES 

Participants 

Fourteen  paid  participants,  eight  male  and  six  female,  ranging  in  age  from  18-26  years  old  served  as 
operators  in  this  task.  All  operators  had  normal  hearing,  as  well  as  normal  or  corrected-to-normal  vision. 

Design 

A  within-subject  design  study  was  employed,  with  two  levels  of  Signal  Difficulty  (Easy  &  Hard)  combined 
factorally  with  four  different  Communication  Formats  (Radio,  3D  Audio,  Chat,  &  MMC).  The  Radio  condition 
had  audio  from  all  six  channels  presented  monaurally.  The  3D  Audio  condition  spatially  separated  each  audio 
channel,  so  that  each  one  virtually  came  from  a  different  location  in  space  around  the  operator.  For  the  Chat 
condition,  sound  was  not  transmitted.  Instead,  text  transcriptions  of  the  signals  were  displayed  (messages  were 
identical  to  verbal  messages)  for  the  six  channels.  Although  operators  were  reading  these  messages,  they  still 
responded  via  voice  communication.  Finally,  in  the  MMC  condition,  both  3D  Audio  and  ASR  capabilities  were 
combined.  The  operators  were  able  to  hear  each  of  the 

channels  localized  in  space  and  see  the  speech-to-text  transcription  of  all  audio  signals.  In  addition,  several 
other  tools  were  made  available  to  the  operators,  including  the  playback,  find,  mute,  and  isolate  functions, 
while  the  highlighting  of  keywords  was  disabled  during  the  study. 

Apparatus 

In  all  experimental  conditions,  operators  were  presented  with  a  27-rnin  communication-monitoring  task  in 
which  they  were  required  to  simultaneously  monitor  six  radio/chat  channels  for  the  occurrence  of  critical 
phrases  from  the  “Cowboy”  call  sign.  The  critical  and  neutral  phrases  for  each  call  sign  were  recorded  by  six 
different  voices  (four  male  and  two  female).  Critical  signals  were  defined  by  the  presence  of  the  word  “Hostile” 
from  the  “Cowboy”  call  sign.  In  the  Easy  condition,  the  critical  phrase  contained  the  “Cowboy”  call  sign 
followed  by  the  phrase  “ID  hostile”  (i.e.  “Cowboy  one  ID  hostile”).  The  critical  phase  for  the  Hard  condition 
was  the  “Cowboy”  call  sign,  followed  by  the  word  “Hostile”  and  location  information  (i.e.  “Cowboy  one  hostile 
north  lead  group  25  miles”).  The  neutral  phrases  were  modified  phrases  from  the  Air  Force  Tactics, 

Techniques,  and  Procedures  communication  brevity  document. 

Phrases  for  each  of  the  six  channels  were  updated  independently  from  each  other  with  the  restrictions  of  30 
messages  per  min  and  that  they  would  be  equally  distributed  across  channels  (810  overall  events;  5 
events/min/channel).  Neutral  signals  for  each  channel  were  generated  at  random  over  a  range  of  10  -  30 
seconds,  with  the  restriction  that  there  were  approximately  4.6  neutral  signals  per  channel  per  minute  (126 
neutral  signals/channel).  The  message  length  for  the  neutral  signals  ranged  from  1-8  sec  in  length  (M  =  3.85). 
Critical  signals  were  presented  at  random  over  a  range  of  10  -  60  seconds,  with  the  restrictions  that  two  critical 
signals  were  generated  per  minute  and  occurred  equally  often  across  the  six  channels  (9  critical  signals/channel; 
signal  probability  =  .067).  The  message  lengths  for  the  Easy  critical  signals  were  1  sec  long  while  the  Hard 
signals  ranged  from  3-5  sec  with  an  average  of  4.3  sec.  In  all  conditions,  observers  responded  to  a  critical 
signal  by  pressing  the  “push-to-talk”  button  on  the  correct  channel  and  repeating  the  message  back.  Responses 
occurring  within  50  sec  of  a  critical  signal  were  recorded  and  all  responses  beyond  were  considered  a  missed 
signal. 
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Procedure 


All  operators  took  part  in  the  computer-based  training,  which  explained  that  their  task  was  to  monitor 
communication  channels  for  the  occurrence  of  a  hostile  entity  that  required  a  response.  The  training,  which 
was  comprised  of  two  separate  60  min  sessions,  allowed  operators  to  familiarize  themselves  with  the  different 
communication  interfaces  and  the  task  itself.  The  order  in  which  operators  experienced  the  eight  experimental 
conditions  was  determined  by  a  Latin  Square.  Data  were  collected  in  a  series  of  sessions,  with  each  session 
lasting  approximately  30  min.  Each  operator  typically  observed  one  session  a  day,  until  the  completion  of  all 
eight  conditions.  Operators  were  stationed  at  a  computer,  where  they  experienced  the  presentation  of  neutral 
and  critical  signals  through  the  communication  interface.  Headphones  were  used  to  present  the  auditory 
signals  and  responses  were  recorded  through  an  attached  microphone.  Immediately  following  each  session, 
observers  completed  the  NASA-T ask  Load  Index  (NASA-TLX;  Hart  &  Staveland,  1988)  which  is  a  validated 
measure  of  perceived  mental  workload. 


4.0  RESULTS  AND  DISCUSSION 

Correct  Detections.  Mean  percentage  of  correct  detections  for  all  combinations  of  response  Difficulty  and 
Comm.  Format  are  presented  in  Figure  2. 


Radio  3D  Chat  MMC 

Comm.  Format 


Figure  2.  Percentage  of  correct  detection  for  all  combinations  of  task  Difficulty  and 
Comm.  Formats.  Error  bars  are  standard  errors. 

Data  from  Figure  2  were  tested  for  statistical  significance  by  means  of  a  2  (Difficulty)  x  4  (Comm.  Format) 
within  subjects  analysis  of  variance  (ANOVA).  A  significant  main  effect  was  found  for  Difficulty,  F  (1,  13)  = 
21.11,  p  <  .05;  the  overall  level  of  detections  was  greater  in  the  Easy  (M  =  90.9)  then  in  the  Hard  condition 
( M=  82.9).  A  significant  main  effect  was  also  found  for  Comm.  Format,  F  (1.81,  23.51)  =  27.35, p  <  .05. 

Post  hoc  Tukey  test  found  that  the  MMC  ( M=  95.8)  and  Chat  (M=  95.4)  formats  did  not  differ  significantly 
from  each  other,  the  scores  for  both  were  significantly  higher  than  those  in  the  3D  Audio  ( M  =  83.0)  and 
Radio  ( M  =  73.3)  formats.  In  addition,  detection  scores  were  significantly  higher  in  the  3D  Audio  format  than 
in  the  Radio  format. 

Along  with  the  significant  main  effects,  the  Difficulty  x  Comm.  Format  interaction  in  the  ANOVA  was  also 
significant,  F  (1.68,  21.79)  =  3.68 ,p<  .05.  To  explore  this  interaction  Tukey  tests  were  performed  on  the 
differences  between  Difficulty  levels  within  each  Comm.  Format.  Detection  scores  were  significantly  greater  in 
the  Easy  than  Hard  condition  for  the  Radio  and  3D  Audio  formats;  there  were  no  significant  differences  in 
difficulty  in  the  Chat  and  MMC  format.  In  this  and  all  subsequent  ANOVAs,  Box’s  epsilon  was  used  to  correct 
for  violations  of  the  sphericity  assumption  (Maxwell  &  Delaney,  2004). 
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Verbal  Response  Accuracy.  Mean  verbal  response  accuracy  scores  for  all  combinations  of  response  Difficulty 
and  Comm.  Format  are  presented  in  Figure  3.  Accuracy  scores  were  calculated  as  percentages  of  the  number  of 
words  in  the  verbal  response  that  were  identically  matched  to  the  words  in  the  critical  phrase. 


■  Easy 

■  Hard 


Radio  3D  Chat  MMC 

Comm.  Format 

Figure  3.  Verbal  response  accuracy  scores  for  difficulty  and  comm,  formats.  Error  bars  are  standard  errors. 


A  2  (Difficulty)  x  4  (Comm.  Format)  within  subjects  ANOVA  revealed  a  significant  main  effect  for  Difficulty, 
F  (1,  13)  =  22.61  ,p  <  .05;  verbal  response  accuracy  in  the  Easy  condition  (M=  99.6)  was  greater  than  in  the 
Hard  condition  (M=  97.5).  A  significant  main  effect  was  also  found  for  Comm.  Format,  F  ( 1.56,  20.29)  = 
22.83, p  <  .05.  Supplementary  Tukey  tests  indicated  that  while  the  verbal 

response  accuracy  scores  in  the  MMC  (M=  99.8)  and  Chat  (M=  99.9)  formats  did  not  differ  significantly  from 
each  other,  the  scores  in  both  of  these  formats  were  significantly  greater  than  those  in  the  Radio  (M  =  96.4)  and 
3D  audio  M  =  98. 1)  formats  which  in  turn,  did  not  differ  significantly  from  each  other. 

In  addition  to  these  main  effects,  the  interaction  between  Difficulty  x  Comm.  Format  was  also  significant,  F 
(1.37,  17.84)  =  1 5.09,  <  .05.  Similar  to  the  correct  detection  results,  post  hoc  Tukey  tests  revealed  that  verbal 

response  accuracy  was  greater  for  the  Easy  than  Hard  phrases  in  the  Radio  and  3D  Audio  formats  but  that  there 
were  no  differences  in  difficulty  for  the  Chat  and  MMC  conditions. 

Reaction  Time.  Response  times  for  the  detected  critical  signals  for  all  combinations  of  response  Difficulty  and 
Comm.  Format  are  presented  in  Table  4. 


Radio  3D  Chat  MlvIC 

Comm.  Format 

Figure  4.  Mean  response  times  in  sec  for  all  combinations  of  task  difficulty  and  comm,  format.  Error  bars  are  standard  errors. 


A  2  (Difficulty)  x  4  (Comm.  Format)  within  subjects  ANOVA  of  the  response  time  data  found  a  significant 
main  effect  for  Difficulty,  F  (1,  13)  =  35. 1 2,  p  <  .05;  response  time  was  faster  the  Easy  (M=  2.82  sec)  than 
the  Hard  condition  ( M  =  4.70  sec).  A  significant  main  effect  was  also  found  for  Comm.  Format,  F  (1 .68, 
21.85)  =  34.85,  p  <  .05,  Supplementary  Tukey  tests  indicated  that  response  times  for  the  3D  Audio  (M=  2.31 
sec)  and  Radio  (M=  2.69  sec)  formats  were  equivalent  and  that  in  both  formats,  the  response  times  were  faster 
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than  for  MMC  (M  =  3.47  sec)  and  Chat  (M=  6.56  sec)  formats.  In  addition  response  times  in  the  MMC  format 
were  faster  than  the  Chat  format.  The  interaction  between  Difficulty  and  Comm.  Format  was  not  statistically 
significant,/)  >.05. 

Perceived  Mental  Workload.  Mean  global  workload  scores  on  the  NASA-  TLX  for  all  combinations  of  response 
Difficulty  and  Comm.  Format  are  presented  in  Table  5. 


Radio  3D  Chat  MMC 

Comm  Format 

Figure  5.  Mean  NASA-TLX  scores  for  difficulty  and  comm,  format.  Error  bars  are  standard  errors. 

A  2  (Difficulty)  x  4  (Comm.  Format)  within  subjects  ANOVA  revealed  that  perceived  mental  workload  was 
significantly  greater  for  the  Hard  (M  =  45.94)  than  Easy  (M=  39.33)  condition,  F  ( 1 ,  13)  =  16.86,  p  <  .05.  A 
significant  main  effect  was  also  found  for  the  Comm.  Format  condition,  F  (2.48,  32.29)  =  15.24,/)  <  .05. 
Supplementary  Tukey  tests  revealed  that  perceived  mental  workload  was  higher  in  the  Radio  (M  =  49.52)  and 
3D  Audio  (M=  47.38)  formats  than  in  the  Chat  (M=  37.83)  and  MMC  (M  =  35.80)  formats.  The  difference 
between  the  two  highest  workload  conditions  was  not  significant  and  the  same  was  true  of  the  difference 
between  the  two  lowest  workload  conditions.  The  Difficulty  x  Comm.  Format  interaction  lacked  statistical 
significant,/)  >.05. 


5.0  CONCLUSION 

This  study  evaluated  the  effectiveness  of  a  recently  developed  tool  by  researchers  at  AFRL’s  Battlespace 
Acoustic  Branch,  to  aid  C2  operators  in  managing  communication  data.  The  assumption  was  that  the  MMC 
suite  provides  operators  with  net-centric,  collaborative  tools  to  accurately  monitor  multiple  communication 
channels  and  make  quick  and  effective  decisions.  The  study  was  designed  to  evaluate  different  communication 
tools  in  their  ability  to  aid  operators  in  monitoring  and  replying  to  critical  messages.  Operators’  ability  to 
detect  critical  messages  and  to  quickly  and  accurately  reply  to  those  messages  was  used  to  evaluate  their 
performance  with  each  of  the  Comm.  Format  with  two  levels  of  Difficulties. 

Operators  detected  significantly  more  messages  using  MMC  and  Chat  than  Radio  and  3D  Audio,  however  in 
support  of  previous  research,  operators  detected  more  messages  with  3D  Audio  than  Radio  (Ericson,  Brungart, 
&  Simpson,  2004).  There  were  no  differences  between  the  Easy  and  Hard  messages  for  the  MMC  and  Chat 
conditions,  however  there  were  more  detections  for  the  Easy  than  Hard  messages  for  Radio  and  3D-Audio.  This 
suggests  that  the  persistent  nature  of  the  messages  aids  in  the  monitoring  of  communication,  independent  of 
message  difficulty.  The  accuracy  of  the  responses  to  the  detected  messages  was  greater  for  the  Easy  than  Hard 
phrases,  as  well  as  greater  for  the  MMC  and  Chat  formats  as  compared  to  Radio  and  3D-Audio  formats.  The 
interaction  between  signal  Difficulty  and  Comm.  Format  for  response  accuracy  was  similar  to  message 
detection.  Response  accuracy  for  the  Easy  phrases  was  greater  than  the  Hard  phrases  for  the  Radio  and  30- 
Audio  condition,  however  there  were  no  differences  between  message  difficulty  for  MMC  or  Chat.  Again,  this 
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suggests  persistent  information  aids  participants  in  accurately  recalling  messages  as  compared  to  perishable, 
auditory  signals.  The  third  measure  of  operator  performance  was  response  time.  Operators’  replies  were  faster 
for  the  Easy  than  Hard  phrases  and  for  the  Radio  and  3D-  Audio  formats  as  compared  to  MMC  format,  which 
was  also  faster  than  Chat  format.  Along  with  the  performance  data,  scores  from  the  NASA-TLX  support  the 
belief  that  MMC  reduces  the  perceived  mental  workload  of  a  communication-  monitoring  task  over  Radio  and 
3D  Audio. 

The  two  most  common  types  of  communication  systems  in  current  use  are  Radio  and  Chat.  The  results  shown 
here  clearly  indicate  that  messages  were  easier  to  detect  and  could  be  read  back  more  accurately  with  a  Chat 
system,  but  listeners  were  able  to  respond  more  rapidly  to  critical  information  with  a  Radio  display.  Thus,  in 
choosing  to  present  information  through  either  a  Chat  or  a  Radio  display,  there  is  a  clear  speed/accuracy 
tradeoff  between  these  two  alternatives.  MMC  offers  a  compromise  between  these  two  extremes  -  it  provides  a 
level  of  accuracy  comparable  to  that  achieved  with  a  Chat  display,  but  with  a  much  faster  response  time.  The 
MMC  suite  not  only  utilizes  the  speed  of  Radio  listening  but  also  increases  the  intelligibility  of  the  audio  signals 
by  spatializing  them,  as  well  as  providing  the  necessary  tools  to  find,  replay,  annotate,  or  edit  messages  that 
were  once  perishable  or  hard  to  find. 

Another  important  finding  was  that  there  were  no  differences  in  correct  detections  and  response  accuracy  for 
difficulty  in  the  Chat  and  MMC  conditions  whereas  there  were  differences  in  the  Radio  and  3D  Audio 
conditions.  This  suggests  the  availability  to  find  and  read  messages  aids  in  the  monitoring  of  communication, 
independent  of  message  complexity.  Situations  such  as  the  ones  confronting  C2  operators,  where  the 
transmission  of  complex  data  is  of  the  utmost  importance,  the  availability  of  persistence  transcriptions  of  Radio 
communication  and  the  ability  to  easily  retrieve  pertinent  information  could  be  a  matter  of  mission  success  or 
failure. 

A  key  limitation  of  this  study  is  that  it  used  a  simple  detection  task  to  evaluate  a  communication  tool  that 
would  be  used  for  more  complex  communication  environments  where  operators  would  also  have  to  integrate 
information  from  other  sources  such  as  tactical  visual  displays.  Studies,  however,  have  shown  that  tasks  that 
require  operators  to  compile  data  from  multiple  visual  displays  to  properly  perform  multiple  tasks,  often  result 
in  poor  management  and  suboptimal  performance  as  operators  will  focus  the  majority  of  their  time  on  the  Chat 
display,  rather  than  the  more  important  tactical  display  (Cummings  &  Guerlain,  2004).  In  theory,  MMC  will 
help  combat  this  issue  and  provide  advantage  over  Chat  because  it  utilizes  multiple  modalities  so  when  an 
operator  must  focus  on  a  visual  display  the  communication  data  is  still  presented  aurally.  Future  studies  will 
examine  this  as  well  as  direct  testing  of  the  MMC  format  on  the  operator’s  situation  awareness  and  use  in  team 
tasks.  MMC  is  continually  being  modified  to  meet  the  warfighters  needs  thus  continuation  of  the  research  will 
explore  further  capabilities  and  interface  designs. 

Although  this  was  a  simple  message  detection  task,  there  is  evidence  the  integration  of  communication  tools 
aids  operators  in  their  ability  to  monitor  multiple  communication  channels.  These  results  suggest  that  MMC 
provides  a  balance  between  the  speed  of  radio  listening  and  the  increased  intelligibility  of  3D  Audio  display 
with  the  data  capturing  capability  of  text  only  displays. 
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7.0  LIST  OF  ACRONYMS,  ABBREVIATIONS,  AND  SYMBOLS 


3D 

Three  Dimensional 

ANOVA 

Analysis  of  Variables 

ASR 

Automatic  Speech  Recognition 

C2 

Command  and  Control 

Comm 

Communication 

ID 

Identification 

MMC 

Multi-Modal  Communications 

NASA 

National  Aeronautics  and  Space  Administration 

NASA-TLX 

National  Aeronautics  and  Space  Administration  -  Task  Load  Index 

Sec 

Second 
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